Method, controller, and computer program product for reducing oscillations in a technical system

ABSTRACT

For reducing oscillations in a technical system plurality of different controller settings for the technical system is received. For a respective controller setting signal representing a time series of operational data of the technical system controlled by the respective controller setting is received, the signal is processed, whereby the processing includes a transformation into a frequency domain, and an entropy value of the processed signal is determined. Depending on the determined entropy values a controller setting from the plurality of controller settings is selected, and the selected controller setting is output for configuring the technical system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to PCT Application No.PCT/EP2017/080192, having a filing date of Nov. 23, 2017, the entirecontents of which are hereby incorporated by reference.

FIELD OF TECHNOLOGY

Complex technical systems like gas turbines, wind turbines, combustionengines, production plants, 3D printers, or power grids usually requiresophisticated control strategies or control policies in order to operatethe technical system in a productive and stable operating range. In manycases, complex dynamical interactions between various components of thetechnical system and/or its controllers induce oscillations in thetechnical system. Such oscillations, however, generally impair theefficiency of the technical system and/or increase its wear.

BACKGROUND

Contemporary controllers for complex technical systems often make use ofmachine learning methods, for example based on artificial neuralnetworks. Such machine learning methods are usually able to modelcomplex dynamical behavior and to provide efficient control policies forcontrolling the technical system. However, in many cases the complexityof such control policies gives rise to complex oscillation patterns,which are difficult to reduce or to detect.

SUMMARY

An aspect relates to a method and a controller for a technical systemthat allow for an efficient reduction of oscillations in the technicalsystem.

For reducing oscillations in a technical system, in particular a gasturbine, a wind turbine, a combustion engine, a production plant, a 3Dprinter, and/or a power grid, a plurality of different controllersettings for the technical system is received. For a respectivecontroller setting a signal representing a time series of operationaldata of the technical system controlled by the respective controllersetting is received, the signal is processed, whereby the processingcomprises a transformation into a frequency domain, and an entropy valueof the processed signal is determined. Depending on the determinedentropy values a controller setting from the plurality of controllersettings is selected, and the selected controller setting is output forconfiguring the technical system.

For executing the inventive method, a controller, a computer programproduct (non-transitory computer readable storage medium havinginstructions, which when executed by a processor, perform actions), anda computer readable storage medium are provided.

The inventive method and the inventive controller may be implemented bymeans of one or more processors, application-specific integratedcircuits (ASIC), digital signal processors (DSP), and/orfield-programmable gate arrays (FPGA).

One advantage of embodiments of the invention can be seen in an abilityto detect and reduce complex and obscured oscillation structures inoperational signals of a technical system, where the signals may besubject to a significant noise level and where only a rough estimate ofexpected oscillation frequencies may be known. In particular, slowoscillations and superimposed oscillations are detectable. The selectionof a particular controller setting from a plurality of differentcontroller settings often allows for a reduction of oscillations withoutaccess to or knowledge about internals of the controller settings, thusseparating a generation of the controller settings from theirassessment. Moreover, the entropy value turns out to be a simple androbust selection criterion in many cases. It often allows assessingseveral hundreds or thousands of different controller settings inacceptable time. The selected controller setting can be permanentlyimplemented in a controller for the technical system, thus configuringand optimizing a control of the technical system.

Particular embodiments of the invention are given by the dependentclaims.

According to an advantageous embodiment of the invention, a controllersetting resulting in a high or maximum entropy value may be selected inthe selection step. A low entropy value of the processed signal in thefrequency domain is usually indicative of oscillations. Hence, byselecting a controller setting with a high or maximum entropy valueunwanted oscillations may be effectively reduced.

According to a further embodiment of the invention, the controllersettings may comprise different control policies for the technicalsystem, the control policies resulting from training one or more controlmodels for the technical system by means of one or more machine learningmethods. Such control policies are often referred to as controlstrategies.

The one or more machine learning methods may employ an artificial neuralnetwork, a recurrent neural network, a convolutional neural network, adeep learning architecture, a reinforcement learning method, anautoencoder, a support vector machine, a data-driven regression model, ak nearest neighbor classifier, and/or a physical model.

According to an advantageous embodiment of the invention, the processingof the signal may comprise determining an autocorrelation of the signal.The autocorrelation specifies a correlation of the signal with itself atpairs of time points i.e., a correlation of the signal with atime-shifted copy of itself. In particular, an autocovariance of thesignal may be determined. An autocovariance specifies a covariance ofthe signal with itself at pairs of time points, and therefore specifiesa correlation of the deviations from the mean value for different timeshifts. By computing the autocovariance or autocorrelation of thesignal, periodic components of the signal are usually amplified comparedto residual noise components of the signal, thus improving the detectionof oscillations.

Advantageously, the autocorrelation or autocovariance of the signal maybe transformed into the frequency domain, by a Fast FourierTransformation (FFT). A resulting frequency spectrum may be normalized,in particular with respect to its total area that is, by an integralnormalization. In the frequency domain, even small periodic componentsof the signal usually show up as peaks.

According to a further embodiment of the invention, the entropy valuesmay be compared with a threshold value, and the selection of thecontroller setting may depend on a comparison result. Before thecomparison, the entropy values may be normalized, in particular withrespect to a maximum possible entropy of the processed signal.

Furthermore, the processing of the signal may comprise subdividing thesignal into time segments of a given length. The transformation into thefrequency domain and the determination of the entropy value may then beperformed for a respective time segment. Accordingly, a determination ofthe autocorrelation, a normalization of the frequency spectrum, and anormalization of the entropy values may be performed specifically for arespective time segment.

Such a time-segment-specific determination of entropy values allows fortracking time-dependent effects.

Advantageously, an estimate of a periodicity of the signal may bereceived, and the length of the time segments may be set depending onthe received estimate. In particular, a value for a maximum periodicityexpected may be received, and the length of the time segment may be setto at least twice the length of that maximum periodicity.

Furthermore, entropy values determined for different time segments maybe aggregated by determining an average value, a maximum value, or aminimum value of these entropy values and/or a quantile value of adistribution of these entropy values. The selection of the controllersetting may then depend on the aggregated entropy values. Such anaggregation often enhances accuracy in case of noisy signals.

BRIEF DESCRIPTION

Some of the embodiments will be described in detail, with reference tothe following figures, wherein like designations denote like members,wherein:

FIG. 1 a machine learning module for training control models for atechnical system; and

FIG. 2 an inventive controller for reducing oscillations in thetechnical system.

DETAILED DESCRIPTION

FIG. 1 illustrates, in schematic representation, a machine learningmodule LRN for training control models CM for a technical system GT. Inthe present embodiment, the technical system GT is a gas turbine.Alternatively, or additionally, the technical system GT may comprise awind turbine, a combustion engine, a production plant, a 3D printer,and/or a power grid.

The gas turbine GT has one or more sensors SE, which measure and provideoperational data OP of the gas turbine GT. Such operational data OD maycomprise sensor data, physical data (e.g., temperature, pressure,entropy, voltage, or current values), chemical data, control data,performance data, status data, condition data, measurement data,environment data, forecast data, simulated operational data, defaultdata, or other data specifying a status or a property of the technicalsystem GT or being produced by the technical system GT or being producedwith regard to the technical system GT. Apart from this, the operationaldata OD may be acquired from other sources, like control modules,simulation modules, or user terminals of the technical system GT or fromother entities related to the technical system GT. According to thepresent embodiment, the operational data OD are transmitted from the gasturbine GT to the machine learning module LRN.

The machine learning module LRN is data-driven and trainable andimplements one or more machine learning methods, in particular areinforcement learning method, for training one or more control modelsCM for the gas turbine GT. Actually, many standard machine learningmethods are available for performing this training. The machine learningmodule LRN comprises one or more processors for executing the machinelearning methods and one or more memory modules, coupled with theprocessor, for storing data used by the machine learning methods.

According to the present embodiment, the machine learning module LRNfurther comprises an artificial neural network NN and an evaluationmodule EM, to which the received operational data OD are supplied.

The neural network NN implements the one or more trainable controlmodels CM, which are to be trained by means of the above mentionedmachine learning methods. The control models CM are aimed to reflect adynamic behavior of the gas turbine GT and its control as accurate aspossible. Each control model CM maps a respective input of operationaldata OD to control signals CS for controlling the gas turbine GT. Eachcontrol model CM should be trained in such a way that the gas turbine GTis controlled by the resulting control signals CS in an optimal way. Thecontrol signals CS or signals derived from these are transmitted fromthe machine learning module LRN to the gas turbine GT for controllingthe gas turbine GT.

The evaluation module EM is used to evaluate a status or anotherproperty of the gas turbine GT depending on the received operationaldata OD according to one or more predetermined criteria. As a result,evaluation values EV are output. The predetermined criteria are chosenin such a way that the evaluation values EV are a measure of a desiredbehavior of the gas turbine GT. In particular, the evaluation module EMmay determine from the received operational data OD a performance, aload, a pollution, a resource consumption, a wear and/or anotheroperational parameter of the gas turbine GT, and may output this orthese parameters as evaluation values EV.

The control models CM are trained by means of received operational dataOD and the evaluation values EV from the evaluation module EM. Thetraining is aimed at optimizing the evaluation values EV. For thispurpose, the evaluation values EV resulting from controlling the gasturbine GT by the control signals CS, are returned to the neural networkNN as indicated by a dashed arrow in FIG. 1 and the control models CMare adapted in order to optimize the evaluation values EV and thus tooptimize an desired behavior of the gas turbine GT. For the adaptationof the control models CM, many standard machine learning methods may beused.

The control models CM may each be trained under many differentoperational conditions of the gas turbine GT. Then, from eachsuccessfully trained control model CM and for each operationalcondition, a specific control policy CP can be extracted. These controlpolicies CP can be used as controller settings for configuring and/orcontrolling the technical system GT. In practice, several hundreds oftrained control policies CP may be generated in such a way.

The generated control policies CP are output by the machine learningmodule LRN. According to embodiments of the invention, the generatedcontrol policies CP are to be examined with regard to their tendency foroscillations. After that, a specific control policy showing particularlow oscillations is selected from all generated control policies foractually configuring and/or controlling the technical system GT.

FIG. 2 illustrates, in schematic representation, an inventive controllerCTL for reducing oscillations in the technical system GT. Referencesigns in FIG. 2 which are identical to those in FIG. 1 denote the sameentities, which are embodied as described above.

The controller CTL may be part of the technical system GT or implementedat least partially separated from the technical system GT. Thecontroller CTL comprises one or more processors for executing all methodsteps of the controller CTL and one or more memory modules, coupled withthe processor, for storing data used by the controller CM.

The controller CTL receives from the machine learning module LRN amultitude of control policies CP usable as controller settings forconfiguring and/or controlling the technical system GT. The machinelearning module LRN may be implemented separately from the controllerCTL or may be at least partially comprised by it.

For each respective control policy CP of the multitude of controlpolicies, the controller CTL controls the technical system GT by meansof control-policy-specific control signals CS(CP) and respectivelyreceives control-policy-specific operational signals SG(CP) from thetechnical system GT. The operational signals SG(CP) represent arespective time series of control-policy-specific operational data, thelatter being specified as above. Control-policy-specific means that thecontrol signals CS(CP) are generated by means of the respective controlpolicy CP and that the operational signals SG(CP) result fromcontrolling the technical system GT by the respective control policy CP.The operational signals SG(CP) may stem from sensors or other sources ofoperational data of the technical system GT.

The controller CTL further receives an estimate of a maximum periodicityT of the operational signals SG(CP).

The estimate T may be received from a user or from a technicalspecification, or may be determined otherwise.

For processing the operational signals SG(CP) for each respectivecontrol policy CP, the controller CTL comprises a subsampling moduleSUB, a correlation module COR, and a transformation module FFT.Furthermore, the controller CTL comprises an entropy module SM, anaggregation module AGG, a selection module SEL, and an artificial neuralnetwork NN.

The sub sampling module SUB is supplied with the maximum periodicity Tand with the operational signals SG(CP). For each control policy CP, thesubsampling module SUB subdivides a respective operational signal SG(CP)into time segments TS of given length. Such time segments are oftendenoted as subsamples. According to the present embodiment, the lengthis set depending on the maximum periodicity T, in particular to a valuegreater or equal to T or to a value greater or equal to 2*T.

The time segments TS are transmitted from the sub sampling module SUB tothe correlation module COR. The correlation module COR determines anautocovariance function AC of each respective time segment TS for eachrespective control policy CP. The autocovariance function AC may beregarded as a kind of autocorrelation and may be determined as acorrelation of the deviations from the mean value of the respective timesegment TS for different time shifts. By computing the autocovariancefunctions AC of the time segments TS, periodic components within thetime segments TS are usually amplified compared to residual noisecomponents, thus improving the detection of oscillations.

The autocovariance functions AC are transmitted from the correlationmodule COR to the transformation module FFT.

The transformation module FFT transforms each autocovariance function ACinto a frequency domain by performing a discrete Fourier transformationresulting in a frequency spectrum for each respective time segment TSand each respective control policy CP. For this, a standard fast Fouriertransformation procedure may be used. The resulting frequency spectraare each normalized, in particular by their total area i.e., by anintegral normalization. The combination of the above processing steps,in particular applying the autocovariance function and normalizing theresulting frequency spectra increases the robustness and performance ofthe inventive method.

Each normalized frequency spectrum is transmitted as a processed signalPSG from the transformation module FFT to the entropy module SM. Theentropy module SM determines an entropy value S of the processed signalPSG for each respective time segment TS and each respective controlpolicy CP in order to assess an oscillatory tendency of the respectivecontrol policy CP.

A respective entropy value S is determined by first calculating aShannon entropy S0 of a respective processed signal PSG e.g., accordingto

S0=−Σp _(i)*log₁₀(p _(i)),

where p_(i) is an amplitude of the i-th frequency bin in the respectivenormalized frequency spectrum PSG and the sum runs over all bins of thatspectrum. After that, the Shannon entropy S0 is normalized with regardto a maximum possible entropy of a frequency spectrum with the givenlength. The result of that normalization gives the entropy value S,which is a scalable measure of uniformity of the respective normalizedfrequency spectrum PSG. That is, the lower the entropy value S is, themore likely it is that the respective time segment TS exhibits someoscillation pattern.

For each respective time segment TS and each respective control policyCP, the respective entropy value S is transmitted from the entropymodule SM to the aggregation module AGG. The aggregation moduleaggregates the entropy values S for all time segments TS of a respectivecontrol policy CP, in particular by determining an average, a maximum, aminimum, or a quantile of the entropy values S involved. This results inan aggregated entropy value SAVG for each respective control policy CP.The aggregated entropy value SAVG allows an overall assessment of arespective control policy CP by a single scalar measure.

For each respective control policy CP, the aggregated entropy value SAVGis transmitted from the aggregation module AGG to the selection moduleSEL. The latter is used for selecting from the multitude of storedcontrol policies CP one or more control policies SCP which exhibitparticular low oscillations. According to the present embodiment, thecontrol policy with a maximum aggregated entropy value SAVG isdetermined and output as selected control policy SCP.

Alternatively, or additionally, the entropy values S and/or theaggregated entropy values SAVG may be compared with a predefinedthreshold value in order to tag a respective control policy CP and/or arespective time segment TS as oscillatory or not. The selection may thendepend on the comparison results or the tagging.

The selected control policy SCP is transmitted from the selection moduleSEL to the neural network NN and configures a control model of theneural network NN. The neural network NN then generates control signalsCS(SCP) according to the selected control policy SCP and outputs them tothe technical system GT for controlling the technical system GT.

As low entropy values in the frequency domain are usually indicative ofoscillations or time periodicity in the underlying operational signal,the selection and application of a control policy SCP which inducesmaximum entropy effectively reduces oscillations in the technical systemGT.

Although the present invention has been disclosed in the form ofpreferred embodiments and variations thereon, it will be understood thatnumerous additional modifications and variations could be made theretowithout departing from the scope of the invention.

For the sake of clarity, it is to be understood that the use of “a” or“an” throughout this application does not exclude a plurality, and“comprising” does not exclude other steps or elements. The mention of a“unit” or a “module” does not preclude the use of more than one unit ormodule.

1. A method for reducing oscillations in a technical system, the methodcomprising: a) receiving a plurality of different controller settingsfor the technical system, b) for a respective controller setting:receiving a signal representing a time series of operational data of thetechnical system controlled by the respective controller setting,processing the signal, the processing comprising a transformation into afrequency domain, and determining an entropy value of the processedsignal, c) selecting a controller setting from the plurality ofcontroller settings depending on the determined entropy values, and d)outputting the selected controller setting for configuring the technicalsystem.
 2. The method according to claim 1, wherein in the selectionstep a controller setting resulting in a high entropy value is selected.3. The method according to claim 1, wherein the technical systemcomprises a gas turbine, a wind turbine, a combustion engine, aproduction plant, a 3D printer, and/or a power grid.
 4. The methodaccording to claim 1, wherein the controller settings comprise differentcontrol policies for the technical system, the control policiesresulting from training one or more control models the technical systemof one or more machine learning methods.
 5. The method according toclaim 4, wherein the one or more machine learning methods employ anartificial neural network, a recurrent neural network, a convolutionalneural network, a deep learning architecture, a reinforcement learningmethod, an autoencoder, a support vector machine, a data-drivenregression model, a k-nearest neighbor classifier, and a physical model.6. The method according to wherein the processing of the signalcomprises determining an autocorrelation of the signal.
 7. The methodaccording to claim 6, wherein the autocorrelation of the signal istransformed into the frequency domain.
 8. The method according to claim1, wherein the entropy values are compared with a threshold value, andthe selection of the controller setting depends on a comparison result.9. The method according to claim 1, wherein the processing of the signalcomprises subdividing the signal into time segments of a given length,and the transformation into the frequency domain and the determinationof the entropy value are performed for a respective time segment. 10.The method according to claim 9, wherein an estimate of a periodicity ofthe signal is received, and the length of the time segments is setdepending on the received estimate.
 11. The method according to claim 9,wherein entropy values determined for different time segments areaggregated by determining an average value, a maximum value, or aminimum value of these entropy values and/or a quantile value of adistribution of these entropy values, and the selection of thecontroller setting depends on the aggregated entropy values.
 12. Acontroller for reducing oscillations in a technical system, thecontroller being configured to perform a method according to claim 1.13. A computer program product, comprising a computer readable hardwarestorage device having computer readable program code stored therein,said program code executable by a processor of a computer system toimplement a method for reducing oscillations in a technical system, thecomputer program product being adapted to perform a method according toclaim
 1. 14. A computer readable storage medium comprising a computerprogram product according to claim 13.